Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learning

In deep reinforcement learning (RL) tasks, an efficient exploration mechanism should be able to encourage an agent to take actions that lead to less frequent states which may yield higher accumulative future return. However, both knowing about the future and evaluating the frequentness of states are non-trivial tasks, especially for deep RL domains, where a state is represented by high-dimensional image frames. In this paper, we propose a novel informed exploration framework for deep RL, where we build the capability for an RL agent to predict over the future transitions and evaluate the frequentness for the predicted future frames in a meaningful manner. To this end, we train a deep prediction model to predict future frames given a state-action pair, and a convolutional autoencoder model to hash over the seen frames. In addition, to utilize the counts derived from the seen frames to evaluate the frequentness for the predicted frames, we tackle the challenge of matching the predicted future frames and their corresponding seen frames at the latent feature level. In this way, we derive a reliable metric for evaluating the novelty of the future direction pointed by each action, and hence inform the agent to explore the least frequent one

Convergence to diffusion waves for solutions of Euler equations with time-depending damping on quadrant

This paper is concerned with the asymptotic behavior of the solution to the Euler equations with time-depending damping on quadrant $(x,t)\in \mathbb{R}^+\times\mathbb{R}^+$, \begin{equation}\notag \partial_t v - \partial_x u=0, \qquad \partial_t u + \partial_x p(v) =\displaystyle -\frac{\alpha}{(1+t)^\lambda} u, \end{equation} with null-Dirichlet boundary condition or null-Neumann boundary condition on $u$. We show that the corresponding initial-boundary value problem admits a unique global smooth solution which tends time-asymptotically to the nonlinear diffusion wave. Compared with the previous work about Euler equations with constant coefficient damping, studied by Nishihara and Yang (1999, J. Differential Equations, 156, 439-458), and Jiang and Zhu (2009, Discrete Contin. Dyn. Syst., 23, 887-918), we obtain a general result when the initial perturbation belongs to the same space. In addition, our main novelty lies in the facts that the cut-off points of the convergence rates are different from our previous result about the Cauchy problem. Our proof is based on the classical energy method and the analyses of the nonlinear diffusion wave

An Application on Text Classification Based on Granular Computing

Machine learning is the key to text classification, a granular computing approach to machine learning is applied to learning classification rules by considering the two basic issues: concept formation and concept relationships identification. In this paper, we concentrate on the selection of a single granule in each step to construct a granule network. A classification rule induction method is proposed

Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation

Training generative models that can generate high-quality text with sufficient diversity is an important open problem for Natural Language Generation (NLG) community. Recently, generative adversarial models have been applied extensively on text generation tasks, where the adversarially trained generators alleviate the exposure bias experienced by conventional maximum likelihood approaches and result in promising generation quality. However, due to the notorious defect of mode collapse for adversarial training, the adversarially trained generators face a quality-diversity trade-off, i.e., the generator models tend to sacrifice generation diversity severely for increasing generation quality. In this paper, we propose a novel approach which aims to improve the performance of adversarial text generation via efficiently decelerating mode collapse of the adversarial training. To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse. Moreover, instead of engaging the cooperative update for the generator in a principled way, we formulate a meta learning mechanism, where the cooperative update to the generator serves as a high level meta task, with an intuition of ensuring the parameters of the generator after the adversarial update would stay resistant against mode collapse. In the experiment, we demonstrate our proposed approach can efficiently slow down the pace of mode collapse for the adversarial text generators. Overall, our proposed method is able to outperform the baseline approaches with significant margins in terms of both generation quality and diversity in the testified domains

Low Expression of DYRK2 (Dual Specificity Tyrosine Phosphorylation Regulated Kinase 2) Correlates with Poor Prognosis in Colorectal Cancer.

Dual-specificity tyrosine-phosphorylation-regulated kinase 2 (DYRK2) is a member of dual-specificity kinase family, which could phosphorylate both Ser/Thr and Tyr substrates. The role of DYRK2 in human cancer remains controversial. For example, overexpression of DYRK2 predicts a better survival in human non-small cell lung cancer. In contrast, amplification of DYRK2 gene occurs in esophageal/lung adenocarcinoma, implying the role of DYRK2 as a potential oncogene. However, its clinical role in colorectal cancer (CRC) has not been explored. In this study, we analyzed the expression of DYRK2 from Oncomine database and found that DYRK2 level is lower in primary or metastatic CRC compared to adjacent normal colon tissue or non-metastatic CRC, respectively, in 6 colorectal carcinoma data sets. The correlation between DYRK2 expression and clinical outcome in 181 CRC patients was also investigated by real-time PCR and IHC. DYRK2 expression was significantly down-regulated in colorectal cancer tissues compared with adjacent non-tumorous tissues. Functional studies confirmed that DYRK2 inhibited cell invasion and migration in both HCT116 and SW480 cells and functioned as a tumor suppressor in CRC cells. Furthermore, the lower DYRK2 levels were correlated with tumor sites (P = 0.023), advanced clinical stages (P = 0.006) and shorter survival in the advanced clinical stages. Univariate and multivariate analyses indicated that DYRK2 expression was an independent prognostic factor (P < 0.001). Taking all, we concluded that DYRK2 a novel prognostic biomarker of human colorectal cancer